The ROC Toolbox: A toolbox for analyzing receiver-operating characteristics derived from confidence ratings.
نویسندگان
چکیده
Signal-detection theory, and the analysis of receiver-operating characteristics (ROCs), has played a critical role in the development of theories of episodic memory and perception. The purpose of the current paper is to present the ROC Toolbox. This toolbox is a set of functions written in the Matlab programming language that can be used to fit various common signal detection models to ROC data obtained from confidence rating experiments. The goals for developing the ROC Toolbox were to create a tool (1) that is easy to use and easy for researchers to implement with their own data, (2) that can flexibly define models based on varying study parameters, such as the number of response options (e.g., confidence ratings) and experimental conditions, and (3) that provides optimal routines (e.g., Maximum Likelihood estimation) to obtain parameter estimates and numerous goodness-of-fit measures.The ROC toolbox allows for various different confidence scales and currently includes the models commonly used in recognition memory and perception: (1) the unequal variance signal detection (UVSD) model, (2) the dual process signal detection (DPSD) model, and (3) the mixture signal detection (MSD) model. For each model fit to a given data set the ROC toolbox plots summary information about the best fitting model parameters and various goodness-of-fit measures. Here, we present an overview of the ROC Toolbox, illustrate how it can be used to input and analyse real data, and finish with a brief discussion on features that can be added to the toolbox.
منابع مشابه
Receiver Operating Characteristic (ROC) Curve Analysis for Medical Diagnostic Test Evaluation
This review provides the basic principle and rational for ROC analysis of rating and continuous diagnostic test results versus a gold standard. Derived indexes of accuracy, in particular area under the curve (AUC) has a meaningful interpretation for disease classification from healthy subjects. The methods of estimate of AUC and its testing in single diagnostic test and also comparative studies...
متن کاملSignal detection theory, the approach of choice: model-based and distribution-free measures and evaluation.
New and old methods of analyzing two-choice experiments with confidence ratings are evaluated. These include the theory of signal detectability (TSD), Luce's choice theory,nonparametric techniques based on areas under receiver-operating characteristic (ROC) functions, andmethods based on S' and omega, proposed by Balakrishnan and his colleagues. New methods for assessing the bias of a complete ...
متن کاملExploring medical diagnostic performance using interactive, multi-parameter sourced receiver operating characteristic scatter plots
Determining diagnostic criteria for specific disorders is often a tedious task that involves determining optimal diagnostic thresholds for symptoms and biomarkers using receiver-operating characteristic (ROC) statistics. To help this endeavor, we developed softROC, a user-friendly graphic-based tool that lets users visually explore possible ROC tradeoffs. The software requires MATLAB installati...
متن کاملBeyond ROC curvature: Strength effects and response time data support continuous-evidence models of recognition memory.
A classic question in the recognition memory literature is whether retrieval is best described as a continuous-evidence process consistent with signal detection theory (SDT), or a threshold process consistent with many multinomial processing tree (MPT) models. Because receiver operating characteristics (ROCs) based on confidence ratings are typically curved as predicted by SDT, this model has b...
متن کاملType 2 tasks in the theory of signal detectability: discrimination between correct and incorrect decisions.
It has been known for over 40 years that there are two fundamentally different kinds of detection tasks in the theory of signal detectability. The Type 1 task is to distinguish between events defined independently of the observer; the Type 2 task is to distinguish between one's own correct and incorrect decisions about those Type 1 events. For the Type 1 task, the behavior of the detector can b...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Behavior research methods
دوره 49 4 شماره
صفحات -
تاریخ انتشار 2017